CRM Perceived Performance
Timeline: March 2020 - May 2020
Tools: Miro, Zoom, Figma, Principle, Google Form
Table Of Contents
1. Project Overview
2. Research Goals
3. Research Methods
a. Recruitment
b. Interview and Card Sorting
c. Observation
d. Cognitive Walkthrough
4. Usability Testing
5. Research Findings
6. Key Insights
7. Conclusion and Recommendation
1. Project Overview
Perceived performance is a vague, little explored concept that is influenced by a variety of factors including task completion steps, task completion time, and loading treatments, to name a few. Perceived performance is a mix of a user’s expectations, usability, and technical performance speed and potentially a lot of other (subjective) factors. A well designed platform can get high user satisfaction despite delays in the processes, while poorly designed solutions can be perceived as slow despite fast technical speed. The speed at which a task can process is often time limited by physical hardware limitations, especially if tasks require to process or transfer large amounts of data, which is often the case with Salesforce. The science and art of perceived performance is to exploit users inability to accurately judge real performance, or objective clock time, and optimize psychological (or brain time) which is the subjective user’s perception of the passage of time.
Based on our research and information of the most popular CRM Salesforce, many users spend a large amount of their work day (4-8 hours on average) using it. Given that Salesforce is the central platform for many of its users and their interactions with the product makes up the majority of their tasks, a high importance is placed on how satisfied users are with it. As indicated before, satisfaction is highly impacted by performance metrics such as loading time, but also highly depends on how a user thinks the system performs. As a result, perceived performance has a strong influence on the user's satisfaction, and, therefore, eventually also on retention and business-relevant metrics such as customer lifetime value. The goal of this project–to improve the overall experience for users, and determine opportunities for improvement of perception of performance of Salesforce software–has a relevant business impact.
2. Research Goals
Salesforce’s main goals of this project was to get a better understanding of the perception of performance of their product by the user. We specified the following three research goals to guide our research approach:
1. Identification of factors influencing perceived performance
2. Diagnosis of usability issues and opportunities related to identified factors
3. Assessment of different types of loading treatments
3. Research Methods
Our research approach consisted of the following research methods. Each step of the research design informed the next one to some degree.
1. Interviews on perceived performance - with users and non-users to learn how to assess performance
2. Observation of Salesforce users - of users of the platform to understand the user flow
3. Cognitive walkthrough - and task analysis of the platform to identify usability opportunities
4. Usability testing of mockups - of mockups to test different types of loading treatment
a. Recruitment
Before we started recruiting participants for our studies, we came up with a list of potential criteria that are relevant when selecting the ideal candidate for our different research methods.
b. Interviews on Perceived Performance
Participants: 5 interviewees from Sales and Customer Success teams. (2 Salesforce, 3 other CRMs)
Procedure: Identification of tasks with good and bad performance, as well as identification of factors influencing performance including card sorting activities
c. Observation of Salesforce Users
Participants: 3 Salesforce users from Sales and Customer Success teams
Procedure: Users would show us the tasks that they do on their CRM and we would ask users questions such as instances where they felt success or frustration
d. Cognitive Walkthrough
Participants: All 4 team members in the role of UX researchers
Procedure: Walk through action sequences of task scenarios and record issues with focus on factors influencing perceived performance and loading treatments
e. Usability Testing of Mockups
Participants: 6 Salesforce users recruited from the Salesforce Trailhead Community
Procedure: Users walk through two usage scenario mockups, loading a table dashboard and importing a file, for users to compare loading treatments. We created 6 loading treatments for the table dashboard scenario and 3 loading treatments for the file import scenario. Users were given think-out-loud assessments and asked open-ended questions. The mockups were created using Figma and Principle.
4. Key Insights
Based on our research findings, we were able to make three key insights
1. Factors influencing perceived performance and their importance vary across users
2. Salesforce complexity strongly influences users’ perceived performance
3. CRM users are open for novel, creative types of loading treatments
5. Conclusion
Perceived performance is a subjective, complex construct influenced by many factors. Each participant has their own unique experiences and perspectives that change how they perceive a system’s performance. Some of these experiences are based on other CRMs they have used in the past, and other perspectives are based completely on their emotion and feelings. Perceived performance as a concept is far more complex than what our team was able to explore from this project. From our research findings and experiences, we are able to provide three clear recommendations for future research.
1. Test more novel loading treatments for different usage scenarios to improve perceived performance and user experience
2. Large-scale, in-depth research on factors affecting users’ perception of system performance, e.g. related to specific types of tasks or user groups
3. Develop a framework to capture interdependencies of factors influencing users’ perceived performance
